1. Technical Field
The present invention relates generally to an improved data processing system and, in particular, to a method and system for improving data throughput within a data processing system. Specifically, the present invention relates to a method and system for improving performance of storage access and control using cache-coherence.
2. Description of Related Art
Traditionally, symmetric multiprocessors are designed around a common system bus on which all processors and other devices such as memory and I/O are connected by merely making physical contacts to the wires carrying bus signals. This common bus is the pathway for transferring commands and data between devices and also for achieving coherence among the system""s cache and memory. A single-common-bus design remains a popular choice for multiprocessor connectivity because of the simplicity of system organization.
This organization also simplifies the task of achieving coherence among the system""s caches. A command issued by a device gets broadcast to all other system devices simultaneously and in the same clock cycle that the command is placed on the bus. A bus enforces a fixed ordering on all commands placed on it. This order is agreed upon by all devices in the system since they all observe the same commands. The devices can also agree, without special effort, on the final effect of a sequence of commands. This is a major advantage for a single-bus-based multiprocessor.
A single-common-bus design, however, limits the size of the system unless one opts for lower system performance. The limits of technology typically allow only a few devices to be connected on the bus without compromising the speed at which the bus switches and, therefore, the speed at which the system runs. If more master devices, such as processors and I/O agents, are placed on the bus, the bus must switch at slower speeds, which lowers its available bandwidth. Lower bandwidth may increase queuing delays, which result in lowering the utilization of processors and lowering the system performance.
Another serious shortcoming in a single-bus system is the availability of a single data path for transfer of data. This further aggravates queuing delays and contributes to lowering of system performance.
Two broad classes of cache-coherence protocols exist. One is bus-based snooping protocols, wherein all the caches in the system connect to a common bus and snoop on transactions issued on the common bus by other caches and then take appropriate actions to stay mutually coherent. The other class is directory-based protocols, wherein each memory address has a xe2x80x9chomexe2x80x9d site. Whenever a cache accesses that address, a xe2x80x9cdirectoryxe2x80x9d at the home site is updated to store the cache""s identity and the state of the data in it. When it is necessary to update the state of the data in that cache, the home site explicitly sends a message to the cache asking it to take appropriate action.
In terms of implementation and verification complexity, the bus-based snooping protocol is significantly simpler than the directory-based protocol and is the protocol of choice of symmetric multiprocessor (SMP) systems. However, the bus-based snooping protocol is effectively employed in a system with only a small number of processors, usually 2 to 4.
Thus, although a single-system-bus design is the current design choice of preference for implementing coherence protocol, it cannot be employed for a large-way SMP with many processors.
Therefore, it would be advantageous to have a large-way SMP design using bus-based cache-coherence protocols.
A distributed system structure for a large-way, symmetric multiprocessor system using a bus-based cache-coherence protocol is provided. The distributed system structure contains an address switch, multiple memory subsystems, and multiple master devices, either processors, I/O agents, or coherent memory adapters, organized into a set of nodes supported by a node controller. Each of the processors may have multiple caches. The address switch connects to each of the node controllers and to each of the memory subsystems, and each of the memory subsystems connects to the address switch and to each of the node controllers. The node controller receives commands from a master device, communicates with a master device as another master device or as a slave device, and queues commands received from a master device. The node controller has a deterministic delay between latching a snooped command broadcast by the address switch and presenting the command to the master devices on the node controller""s master device buses. The memory subsystems contain a memory controller and a fixed delay pipe from the address port to the memory controller so that the memory subsystem has a deterministic delay between receiving a command from the address switch and presenting the command to the memory controller. The buses between the master devices, the node controllers, the address switch, and the memory subsystems are operable using a variety of bus protocols. The data throughput of a port within the node controller is independent of the total number of ports in the data processing system.